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ABSTRACT 

Social programs are often based on unclearly , 
articulated theoretical rationales. The evaluations of these programs 
are also often relatively theory free, and therefore make limited 
contributions to theory. A Program Development Evaluation (PDE|) 
approach, < intended to integrate theory testing with the development 
and evaluation of action programs, is described and illustrates. This 
approach has been applied for over 2 years in the national evaluation 
of locally developed delinquency programs in 69 schools, and h,4s 
resulted in (a) -the implementation of several true experiments'/, 
conducted in ,collaborat ion with project implemeaters, (b) an increase 
in the clarity of projects' theoretical rationales, and (c) th6* 
identification and measurement of theoretical intermediary variables 
as well as outcome variables. Based on the action research paradigm, 
the PDE approach appears to increase the theoretical and practical 
relevance of evaluation research. (Author/PN) - 
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• , The Center for Social Organization of Schools 

The Center for Social Organization of Schools has Wo primary objec- 
tives- to develop a scientific knowledge of how schools affect their 
students, and to use this knowledge to develop better^ school practices 
arid organization.. 

The Center works through .five programs to achieve its objectives. The 
Studies jn School Desegregation program applies the basic theories of 
sodial organization of schools to study the internal conditions of 
desegregated schools, the feasibility of alternative desegregation poli- 
cies; and" the interrelations of school desegregation .with other equity 
issues such as housing and job desegregation. The School Organ i zat i on 
program is currently concerned with authority-control structures, task 
structures, reward systems, and peer group processes in schools. It has 
produced a large-scale study of the effects of open schools, has devel- 
oped Student Team Learning instructional processes for teaching various 
subjects in elementary and secondary schools, and has produced a compu- 
terized system for school-wide attendance monitoring. The School £CO~ 
cesses and Sarsex Development program is studying transitions f rctn'high 
school to post-secondary institutions and the role of schooling in the 
development of career plans and the actualization of labor market out- 
comes. Tne Studies, ill nelinouencv and School, Env i ronm e nts program is 
examining the role of school environments and experiences on-delirvr 
quency. 

The Center also supports a Fellowships in Education Research program 
that provides opportunities for talented young researchers to conduct 
ernd publish significant research, and to encourage the- participation of 
tfcmen and minorities in research on education. 

This report, prepared by the Delinquency and School Env i ronments pro- 
gram, describes a method to make evaluation research' more . theoretically 
and practically useful. r 
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A Theory-Ridden Approach to Program Evaluation: 
Method for Researqher-Implementer Collaboration 



Abstract 



Social programs .are/ often based on unarticulated or 
• unclearly articulated theoretical rationales. The evaluations 
'of these programs are also often relatively theory free, and 
therefore make limited contributions to theory. A Program . 
' Development Evaluation (PDE) approach, intended to integrate 
' theory testing with the development and evaluation of action 
programs, is described and illustrated. This approach has, 
been applied for over two years in the national evaluation of 
locally developed delinquency programs in 69 schools, and has 
resulted in (a) the implementation of 'several true experiments 
conducted in collaboration with project implementers, (b) anC 
increase in the clarity of .projects' theoretical rationales, 
and, (c) the identification and measurement of theoretical 
intermediary variables as well. as outcome variables. .Based on 
the action research paradigm, the IDE approach appears to 
increase the theoretical and practical relevance of evaluation 
research. • > 



This repott is about a critical 
task facing behavioral scien- — . 
tists--the task of designing 
research that advances both theory 
and practice. Educational and. 
social programs are of ten based on 
unacticulated or uncTearly articu- 
lated theoretical rationales. The 
evaluations of these programs are 
also often relatively theory free, 
and therefore make limited contri- 
butions to theory* ( My colleagues 
and I (Gottf redson, Note 1) have 
created a Program Development 
Evaluation (PDE) method intended 
to make possible the implementa- 
tion and testing of strbnger, more 
theoretically guided action 'pro- 
grams. T&is approach has been 
applied fcfr over two years in the 
national evaluation of locally 
developed delinquency prevention 
programs irf 69 schools, and has 
resulted in (a) th£ implementation 
of several -true experiments con- 
ducted in co^.aborat ion 'with proj- 
ect implementers, (b) an increase 



in the clarity of projects' Itheo- > 
retical rationales, and (c) the 
icterrtif ication anc^ measurement of 
'theoretical intermediary, variables 
-as well as outcome variables. 
Based on the action research para- 
digm, the IDE approach appears to 
increase the theoretical and prac- 
tical relevance of .evaluation 
research. 

The IDE method has evolved from * 
efforts to help school systeirfs and 
community-based organizations create 
effective programs and to evaluate 
those programs. A primary task 
for behavioral scientists is the 
development of know lege, but 
researchers who have worked with ' 
practitioners struggling to solve* 
problems know that the 'behavioral 
sciences have more to offer than 
the techniques of research, ar"» v 
smcngasboard of previously ipsted 
interventions, and a handfiH.1 of 
theories." Knowledge about the • 
ways organizations behave and 
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Theory and Evaluation ' 

about the psychology of decision 
making and planning in groups can 
be used to assist organizations 
not only to- examine the results of 
their efforts , but also to 
increase organizational effective- 
ness in adopting theoretically 
plausible innovations and assuring 
sir faithful implementation.* 



The K>E method provides a 
structure to merge the roles and 
activities of organization ctevelj- 
oper, theoretician f and evaluation, 
researcher, Shis method is a form 
of action research (Chein f Coojc, & 
Harding, 1948; Lew.in, 1947/ San- 
ford, 1970) . The approach assumes 
that the prospects for promoting 
change are greatest when organiza- 
tional decisionmakers' stake in 
the research is irede cleat by 
their own participation in the 
research. Decision makers and 
researchers collaborate through a 
continuing dialogue in which 
researchers provide feedback on 
the consequences of project 
action. Action research 1 involves 
t a cycle of hypothesis formulation 
and planning, action, evaluation 
and information feedback, and then 
renewed hypothesis^ormulation and 
planning. As > the cycle is * ■ 
repeated, and information derived 
f rap project efforts 'and research 
is used in decision making, jroj- 
ects should become nore effec- 
tive — turning the process into an 
napwarti spiral of 'activity. And as 
the cycle is repeated, theory 
testing' is refined, resulting ih 
better; more practical ttfeory. „ 

> Projects usually change cv6r ■ - 
Jtfcte <xx the basis of the expedi- 
ence gained as 'th>e£ develop (Wil- 
kins & Gottf redson, Note 2) . What 
Pearl (tJote 3)*, has called "quality 
control w is rfeeded to insure not 
just that a program is run accord- 
ing* to 1M plan, but that a plan 
exists and is modified to coincide 
with the way a project, as it \ 



develops, is actually run. Mary 
attenpts to demonstrate the effec- 
tiveness of specifiable social 
programs have failed in part 
because plausible interventions 
were not implemented of their 
implementation was not documented 
(Quay, 1977; Sechrest, White, & 
Brown, 1979; Hall & Loucks, 1977), 
or the plan for the innovation was 
not clearly articulated at the^ 
outset (Sarason, 1971Jt. 

The IDE method facilitates the 
'study of the development pf a pro- 
gram by assisting in the planning 
process. It provides a mechahisnr 
by which an organization can make 
its plans and the thepretical ■ 
rationale behind them explicit, 
and then engage in theory- relevant 
research as it goes about imple- 
menting and studying its activi- , 
ties. It also helps the 
researcher monitop and document 
project implementation as the 
project evolves. In short; • the 
PEE method is an attempt to inte- 
J grate theoretical research yith 
evaluation and organizational 
development. , Its action* research 
approach to knowledge generation 
and organizational .growth is 
derived, from a tradition of con- 1 
cern for practical theory, useful 
research*- 'and organizational 
change and development. 



Guiding. Principles 
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The evolution. and practical use 
of the EDE method* is guided ,by fc 
s , seyen general principles. 'Ihese 
4 principles *are used" to, ireke deci- 
; , sifcns about the conduct of the , 
evaluation research and to j^solve 
ambiguities about appropriate next 
steps. Essentially, these guiding 
principles are a theory about 
doing theory-relevant research 
while increasing organizational 
effectiveness. Evaluation 
research conducted in accordance 
with these seven- principles ^ould 
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be roost "productive. £indiarly f 
organizations, that implement pro- 
grams using these principles 
shouid be more effective that 
organizations that ignore them* 

1. Acticai projects guided by 
, explicit, theories that can be 

translated into practice will be 
nest effective. Not all theories 
can be translated into action by 
any given project f and all theo- 
ries are not created equal. Sci- 
ence progresses by creating ideas 
and eventually rejecting the 
unproductive ones; some ideas have 
shown utility, and other things 
'being equal f ideas that have sur- 
vived previous empirical tests 
should have considerable prece- ( . 
dence. The theory guiding a proj- 
ect should be a teirplate for it f 
with decisions and interventions 
judged against it at the same time 
that the theory ,is judged by the 
effectiveness of the interven- 
tions. Projects will ,-be imple- 
mented with mofet enthusiasm f be 
strongest, and contribute most to 
" Knowledge if (a) £he ,theory is 
generated by or regarded as sensi- 
ble by the project iirplementers 
themselves f and ^b) th? theory 
accdrds with evidence from previ- " 
oui|^search and evaluation. 

2 . Infective frdoptiori of an 

. innovation or other ^interventipn 
is more likely wh%n explicit plans 
for adoption: are available/ and 
perceived obstacles to organiza- 
tional change are seen as likely 
to be Overcome by a asj^ceivable 
plan. , ■ ■ ' "~ - 

_.. 3 V . Effective implementation of 
an intervention or - innovation is * 
more likely if (a) blueprints for 

"th&^ifitervention (itenuals, proto- 
cols'/ etc;) are available , or if 
the intervention is structured by 
forms, rules, or operating proce- 
dures; and (b) incrementation is*" 
subject to data, guidance , i.e., 



Theory .and Evaluation 

observation and feedback of infor- 
mation to workers about the degree 
to which their behavior accords 
with the behavior specified by the 
blueprints for the intervention. 
-Effective blueprints include p&jans 
for data guidance , and provide for 
t documentation of the implementa- 
tion df interventions. 



\ 



4. Projects will increase in 
effectiveness under evaluation 
pressure. This pressure takes 
many forms , the most important of 
which are: (a) pressure to focus 
on theory in examining organiza- 
tional behavior and the behavior 
of the organization's inhabitants; 

(b) pressure from potentially use- 
ful knowledge or information of t 
relevance. to the organization; '* 

(c) pressure from "personal know- 
ledge" based on many sources, * 
including direct observation' or 
experience; (d) pressure from the 
rigorous, theory-based evaluatidn 
of intervention conponents; , 

(e) pressure' from the rigorous, 
theory-based evaluation of proj- 
ects as a whole; (f) pressure from 
feedback about steps taken to , 
adopt an innovation; and „i„ 
(g) pressure from feedback about 
steps taken, to implement an inter- 
vention. ' . 

5. Projects internalizing 
these principles will behave in 
accordance witi) them more often 
than projects that sinply comply 
with the application of Jthem, and 
the former will therefore ulti- 
mately be more effective. 

1>T ^eytoore directly project 
impiementers benefit from evalua- 
tion, the more evaluation will be 
integrated with project opera- 
tions.. * 

„ 7. The interests of project 
implementers and evaluators coin- 
cide because *ori& of the bestVays' 
to create ccrnmunicable knowledge 
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Theory and Evaluation 

is' through the rigorous f theo^ , 
guided evaluation of well imple- 
mented interventions that can be 
described so well that others can 
understand what was done and 
therefore replicate them. Both 
inplementers and researchers heed 
rigorous evaluation, the adoption 
of innovations , well* implemented 
interventions, thorough descrip- 
tion, and theory. 

These guiding principles appear 
to us to accord with sound organi- 
zation development practice 
(French & Bell, 1978), sound field 
research practices (Empey, 1980) , 
and practical Wisdom in evaluation 
research (Tharp & Gallimore, Note 
4) . They were used to create the 

PDE method. 

* *• * 

Organization Development as SH 
Antecedent q£ flffi 

One of the roots of Program 
Development t Evaluation is the 
practice of organization develop-, 
ment (OD) . French and..Bell (1978) 
characterize OD as a process 
involving action research that 
emphasizes normative change, is 
based in behavioral science, N 
involves experience-based learning 
of intact work te^ms, and empha- 
sizes goals and objectives. By 
characterizing OR as a process, v 
French and Bell mean that CD is 1 

' "not to be regarded as a 
one-shot solution to organi-r 

- zational problems, but -more 
as a 'growing toward 1 
greater effectiveness 
through a .series of inter- 
vention activities wejr fa 
period of tiite. . . . \Qiang- 
ing the culture of. . ; . an 
entire organization is a 
long-term, involved process" 
(p. 69) . 

v In addition, they see CD as a pro- 
cess involving rational,, empirical 



strategies, but, one that is even' 9 
more* dependent On normative-reedu- 
cative strategies: "The sXieilfc 
defines what changes and improve- t 
ments he or she wants to make, 
rather, than the change agent; the 
cnange agent attenpts to intervene 
in a nutual, collaborative way 
with . the client as they together 
define problems and seek solu- 
, tions; anything hindering ef fee- ' 
- tive problem solving is brought to 
light and publicly examined" (pp. 
75-76) . The emphasis on normative 
education is based on the assump- 
tion that behaviors are rooted in 
norms, values, or beliefs as well 
as in rationality and self-inter- 
ests CD is a data-bas^d approach 
to planned change in which infor- 
mation is a spur to action. 
Unpleasant information is not to 
be avoided but rather treasured 
because it may lead to advance- 
,« ment, to clarification of prob- 
lems. Typically, CD emphasizes 
concrete goal settirig through the 
shared experience of a group in 
-formulating plans. The on-the-job 
learning experience of an intact 
group is presumed fcp promote 
organizational and individual 
effectiveness. 

The interactive, collaborative, 
participative approach, often used 
by behavioral scientists or' CD 
specialists serving as consultants 
or facilitators of organizational 
planning and decision making, has 
much to offer in overcoming some 
of the difficulties a research 
effort may expect to face. First, 
increasing an organization's 
effectiveness should increase the 
likelihood that it will succeed in 
inplementing interventions with a 
•possibility of being shown to be' 
.effective when subjected to seri- 
ous summative evaluation. \ Second, 
in the OD process, the scientist > 
approaches an organization in a 
iflannerthat may decrease the 
extent to which he or she is 



fcerceiyed as an alien invgder. By 
helping an organization clarify 
itsjgoals and objectives , by 
assisting in creating open oonmu- 
nication about problems, and ky 
fostering the expectation that 
projects will change and develop 
over time, the researcher nay (Some 
to be considered more as an 
insider f an entity to be trusted 
to convey useful news, And, the 
perspective that information , eveft 
uncfomf or table infornation f is 
valuable in fostering grcwth and 
confronting important problems' nay 
decrease the organization's usual 
,fear of evaluation. Finally, the 
links between CD and action 
research make the interjection of 
formal research possible. 

The Program Development Evalua- 
tion method 4 is in part a descen- 
ded of an CD method previously 
used by the Social Action JRe search 
Center (Blanton & Alley, Note 5) , 
in a series of projects to manage 
^and study social change. This 
1 predecessor , called the Program' 
. Development (PD) model/ was devel- 
oped through attempts to evaluate 
human service projects. In the ID 
model f feedback is a mechanism of 
pioject development that involves 
monitoring a project's environ-' 
ment f the implementation of stra- 
tegies , and the achievement of 
goals. In practice, the Program 
Development specialist focuses on 
interaction witif project imple- 
mented to assist in assessing 
needs, in articulating goals and 
more specific objectives r in. ana- 
lyzing a project's forcefield 1 ' 
(environmental constraints and 
resources) , and in developing v 
strategies for change or implefoen- 
tation. like other forme of CD, 
ED emphasized participatory plan- 
ning in part to foster rormative-v^ 
reeducation and in part to 
increase organizational and indi- 
vidual competencies in decision 
making and planning. 



Theory and/ Evaluation 
Action Eefigftcsh as an Antecedent 

PEE has its roots, in action 
xeseatch. According to French and 
Bell (197fi|) f the' origins of action 
research lie in the work .of Dewey 
(1933), Collier (1945), and Lewin 
(1946).,, The roots of action 
research are f however , deeper than 
this, ttiey can be traced back to 
the Baconian formulation of the 
scientific method, which specified 
thr^e* steps: (a) the formation of 
hypotheses, (b) the empirical 
testing of the hypotheses, and 
(c) the acceptance or, rejection of 
the hypotheses (Deese, 1972).. : 
Action is taken to "twist the 
lion's tail" to learn about 
nature. Since Bacon, science has 
been active rather 'than specula- 
tive,, historical, <k 'reflective. 
Dewey translated jbhe scientific 
method of problem solving for lay- 
persons, and Collier and Lewin 
both applied the scientific method 
to solving practical social prob- 
lems. v . 4 ' / • " 

Collier (cited in French fit, 
Bell, 1978), a commissioner o£ 
Indian Affairs concerned with 
improving race relations, wrote of 
action research, claiming that: 

) 

Research and thai nore , 
research is essential to the 
program, that in the ethnic 
field research can be macte a 
tod of action essential to 
all other tools, indeed that 
it ought to be the master - 
tod.. 'But we had in mind a 
particular kind of research, 
or, if, you will, particular ^ 
conditions.^ Ws had in .mind' 
research impelled frcm oen- 
. tral areas of needed action. - 
. An3 since action is by 
nature not k only specialized 
but also integrative . . . r 
our needed research must be 



"Theory and Evaluation 

of the integrative sort. 
Again, since the findings of 
the research nust be criti*- , 
. . cized by them through their 
experience, the administra- 

1 ' tor and the layman irust • ' 
, themselves participate crea- 
tively in the research , 

. ' impelled as it is from their ' 
*:<xm area of need. » (p. 94) 



Broader attention was called to 
action research by Lewin (1947) , 
an eminent and influential psycho-^ . 1 
logical theorist with a keen 
interest in the application^ of 
psychplpgy. . He saw that coopera- 
tion between the change agent (or 
field worker) and the researcher * 
is important for *both planning and 
management: ' 



Planning starts usually 
with something like a gen- 
eral idea. For one reason 
or another it seems desira- * 
ble to reach a certain 
objective. . . . ttie first 
sttep, then, is to examine 
the idear caiefully in the 
liight of the means 'availa- 
ble. Frequently more fact- 
finding about the situation 
is required. If the' first . 
period of planning is suc- 
cessful , two 'items emerge: 
an 1 overall plan 1 , of* how to . 
reach the objective and a 
^pision in regard' to the 
first step of the action. 
Usually this planning te^ 
also somewhat modified the 
original idea. The next 
period is devoted to execut- 
ing the' first step of the 
overall plan'. . . (and) by 
certain fact-findings. ... 
This . * fact-finding has 
four functions. It should 
evaluate the action by .shew- 
ing whether what has been 
achi^ed is above or belcw 



expectation. ' It .should 
serve as a basis for cor- 
rectly pLanning ; the next . 
step, {for} modifying the 
•overall plan/ Finally, it 
gives the planners a chance 
to learn, that is f to' gather 
hew general insight . '. 
regarding the strength or 
weakness .of certain . . . 
techniques of action. 

Rational social manage- 
ment r therefore, proceeds ih 
a Spiral of steps, each of; 
which 'is composed of a cir- 
cle or planning , action , and 
fact-finding about the 
,r6sult Qf^the action, (pp. 
333^334) . ' 



This sequential and spiral ing 
method of problem solving is now 
widely used ih organizational 
development efforts, and has been 
applied in a variety of indus- 
trial , human service, and educa- 
tional action research projects; 
and it apjaears to be at the heart 
of Tharp and Gallimore 1 s (Note 4) 
Evaluation Succession model. 

Several varieties of activity 
are often called' action research „ 
(Chein, Cook, & Harding, 1948) . 
Sometimes the effort is limited to 
diagnosis and recommendations; ' 
sometimes organizations br project 
inplementers carry out the entire 
process; sometimes records or 
diaries of actions taken and their 
perceived effects are maintained. 
Tharp and Gallimore (Note 4) 
describe several ways of "know- 
ing," each appropriate to differ- 
ent stages in the development of' a 
program.^ What they call "experi- 
mentation,", "qualitative/per spnal 
knowing," "data guidance,". and 
"program evaluation" are all use- 
ful in program development and 
research.. But the variety of 
action research most productive of 

ii . • ■ . . 



trustworthy knowledge is experi- 
mental action research. Unfortu-. 
nately,' experimental .action , , . 
research is- also the most diffi- , 
cult to perform, because it- 
requires the conditions necessary 
for confident inference (Cook & 
Campbell, 1979; Gottfredson, Note 
l,Chap. 3) ,-and a stable set of 
interventions that the organiza- , 
tion knows* how to- and can imple- 
ment in testable form. Seldom do . 
OD efforts aim to implement exper- 
imental action research,, largely 
because it i is so difficult. 
Implementing experimental action 
research is, however, a chief, aim 
of the .Program Development Evalua- 
tion method. Because of the pace 
-of organizational change, rigorous 
examination of the consequences of 
actions may not always be timely 
and less rigorous ways of knowing 
about the effects of innovations 
are "often necessary. But research 
mustSbe coordinate rather than 
subordinate! to problem solving; ^ 
solving problems without learning 
how or why they were solved^will- 
contribute little to organiza- 
tional effectiveness or to theory * 
in the. long run. 

3foe £DE Ufithoji ' 

Hie PDE method emphasizes , 
(a) theory, (b) measurement, and 
(c) experimental or quasi-experi- 
-mental design to a greater extent 
than do many approaches to organi- 
zational change and development. 
In addition, some a>raronly used 
terms (most notably, "objectives") 
are defined in a special way 
Mastery of the practical meanings 
of the terms in the PDE structure 
will provide change agents with a **• 
language for thiHking about, 
facilitating, managing, and study- 
ing their change efforts. 

The Program Development Evalua- 
tion method, illustrated in Figure 
1, incorporates theory as an 
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explicit, coicponent, gives measura- ' , 
bj£ goals and objectives & hard- • 
nosed meaning, and incorporates 
planning for evaluation implemen- 
tation in the same way that plan- 
ning for any other aspect of a , 
project is incorporated.. It 
allows project "implementers and 
evaluators to monitor, critical 
benchmarks -in' the adoption of x any 
strategy to create -change, and it r . " 
allows them to monitor implemertta- ' 
tion standards in the implementa- 
tion of interventions. The ,prin-' 
cipal concepts involved in the PDE 
structure are elaborated below: 

Problems and- Goals ' . • 

' Work in any project Should 
begin with an exploration of its 
intent. Most organizations f and * 
most researchers, have Ttultiple 
aims. Within the PDE framework, . 
an \ overarching aim is called a 
goal, A goal is the obverse of a 
problem;' it specif ies. how the 
level of the problem may be meas- 
ured and therefore how one may 
know if progress is being made. 
Several secondary questions are 
iirportant when, discussing goals, 
The first question, serves to 
reduce ambiguity and enable evalu- 
ation; it asks how each goal may t 
be measured. The second question 
serves to prcn^te realistic # 
research designs; it asks when a ^ 
project can realistically expect * # ■ 
to make a subetantiaft difference ^ 
and therefore specifies the dura- 
tion of- intervention and the tim- 
ing of measurement. 2nd the third 
question, essential in Experimen- 
tal or quasi-experimental action 
research, asks hew one, may knew » 
that the project itself was - 
responsible for progress tcwapds~ * 
the goal. These questions are, of * 
course, steps toward involving 
project inplementers in the design 
of the research. . And, they serve 
to make explicit what the prgani^ 
zation- expects to accomplish. 
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Theory 

Actions, are taken for reasons 
that are ^either ^articulated^ 
unarticulatea. Ihe HDE nethod is 
a vehicle for making theory .expli- 
cit* , 'Ibis is useful because; as 
the Panel on Research on ReKabili- 
>ta^ive Techniques (Martin f 
Sechrest, & Redner, 1981) notes: 

In atteirpting to solve any 
problem , a clear idea of the 
nature of the problem, its 
causes , and developmental 
processes is vital* In the 
absence of -an adequate con- 
ceptual framework . . • the 
rush of enthusiasm- for an . 
interesting intervention is 
likely to short-circuit con- , 
sideration of these factors* 
The result is . . . efforts 
that n^Tbe unrelated to the 
causes oft, crime f ignore the 
most suitable target popula- 
tions, and fail to consider 
questions of optimal timing 
and strength of the inter- 
vention* .The adoption of a 
theoretical : framework neces- 
sarily prompts consideration 
of the above factors and, 
one hopes f thoughtful devel- 
opment and implementation of 
^ i . interventions f thereby 
in^easiha the cfeances £oi— 
* ef f eCtivenessT tpr. 29k 



Theory helps to organize know- 
ledge and to communicate, 1 it pro- 
vides a guide for action, and it 
assists in developing and assess- 
ing interventions* 

"Onc$ a basic problem is 
; stated in theoretical terms, 
placers have an explicit 
foundation on which to build 
an intervention strategy and 
from which to derive a 
research strategy in con- 
junction with the 



) 

intervention" 
al.r 1981, p. 
Glaser, 1980) 



(Martin et 
34; cf. 



In short,, an explicit theory poo- 
vi<3es a tenplate for project 
irplementers 1 use in building 
.their interventions, as well as a 
"tgnplate by which bQth implement- 
ed and researchers can assess 
those interventions. Therefore, 
tlje HDE process calls for deliber- 
ate and careful consideration of 
the question, "Why do these prob- 
lems exist?" When an intervention 
is designed using theory, its 
evaluation tests the theory under- 
girding the intervention. 

Objectives 

In the language of IDE, an 
objective is an intermediary out- 
come that a project's theory, of 
action implies i£ important. Like 

rls, objectives must be stated 
measurable terms. 

Sane examples may help make the. 
distinction between goals and 
objectives clear. Suppose that a 
change agent wishes to* deer ease 
the- death rate due to gastroent;er- 
, itis in a rural society. The 
change agent theorizes that the 
suffering and death ape due to the 
contamination of village water \ 
sufplies^th the diolera microor- 
ganism. This theory might suggest 
a campaign to chlorinate* wells, 
with the Objective of decreasing 
this contamination. The cbjective 
would be measured laboratory 
analyses of well-water sairples to 
determine the levels of microbial 
contamination, .and attainment of ■ 
the goal might be iteasured 
counts of deaths per 100,000 popu- 
lation due to gastroenteritis. 
Another change agent iftight see the 
problem somewhat^differenfely. 
This second change agent may 
theorize that the suffering and 
death are due to poor 
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environmental sanitation: Because 
few villagers use sanitary 
latrines, well vater is easily 
contaminated and the cholera 
microorganism spreads from , , 
infected to, uninfected persons. 
This theory might suggest an envi- 
ronmental sanitation campaign 
directed at persuading villagers 
to 'construct sanitary latrines and 
sanitary wells, The objective now 
involves villager behavior , and 
ipight be measured by the propor- 
tion^ households using sanitary 
latrines and water from protected 
wells. A theory can, of course, 
suggest multiple interventions and 
multiple objectives. The second 
change agent 1 s theory would also 
reasonabl^jjiply chlorination of' 
'wells and^assiessments of well 
water. 1 The more comprehensive a 
theory, QieJroore complex the array 
of interventions and objectives it 
is likely to. suggest. . 

Change agents could develop 
theories at many levbls to explain 
the problem of cholera deaths, and 
* each level would suggest somewhat 
♦different intervlhtions. To con- 
tinue- the examples, change agents ■ 
might attribute the problem to fc 
(a) normatiye beliefs in village 
- -sociefeiesJthat^cur rent standa rds 
. of environmental sanTEaTion.are * ' 
adequate, (b) the poverty and r 
segregation of the rural people,' 
which deprive them of the 
resources to build sanitary 
devices and concentrates them so 
that thqr are at hicjti risk, 
(c) social stratification that 
allows only an elite merchant . 
m class access to sufficient 
a resources to enjoy a sanitary 
'environment/ (d) stratification in 
tfie world system that enables 
capitalist countries tolceep coun- 
tries with rural rubber-tapping 
populations impoverished and the 
cost of raw materials lew. Each 
of these theories may have consid- 
erable validity. Yet each would 



imply different interventions to 
solve the problem, ranging from 
dumping chlorine in wells to over- 
throwing the capitalist world- sys- 
tem. No single cholera prevention 
project is likely to attempt 
interventions at all of these lev- 
els, and so will not have objec- 
tives at each level. A jxcfjeffi's 
theory of action — the theory Jtjfrat 
drives its interventions — ife raie 
theory that is relevant in specif- 
ying objectives. / 

Again, answers to several key 
questions— hew objectives nay be 
measured, when effects are, to be 
expected, and how cne may knew , 
that the intervention caused the 
ef fectsr-serve to create the eval- 
uation or research design. In* 

addition, the explicit statement 
of objectives serves to make clear 
to the organization what it 
ejects to happen- as a result of 
its efforts. 

Intervention 

An intervention is an action ' 
taken to achieve an objective or* 
set of objectives. Ordinarily, it 
is a major component of a project. 
The ±erm is often ^noHymous tfith 
"change, " "treatment," or "compo- 
nent." Some 'interventions are 
aimed at changing the behavior, 
attitudes, or status of individual 
people; others are aimed at chang- 
ing the, behavior of an organize*-. . 
tion or collectivity. An inter- 
vention is a process, action, 
structure, rule, or substance that 
is apgfied or put in place to 
achieve an objective or set of 
objectives, and therefore to move 
cioser to achieving a goal. An 
intervention may be chemical, 
physical, biological, behavioral, 
social > political, or structural. 
The interventions employed shbuld 
be aimed at the objectives an ' , 
organization 1 s t^ispry of* action 
implies nttsfc be achieved."* 

10-16 



Forceffeld 

A forcef ield is the social-psy- 
chological field that immediately 
surrounds a decision or action. 
It includes the forces that compel 
or restrain against alternative 
actions as they are perceived by* 
an individual or corporate aqtor. 
The notion of a forcef ield joom^s 
from Lewin 1 s (1951) ideas about 
the field of forces influencing 
action. An examination or analy- 
sis of an organization's force- 
f ield, especially one that focuses 
on the field** in terms of, the 
resources available and the obsta- 
cles to action, is frequently use- 
ful for four reasons: (a) By 
focusing on the organization's . 
perceptions of environmental 
influences, the nature of these 
perceptions becomes explicit and 
open to, scrutiny, revision, amen- 
dat ion, v supplementation, and test. 

(b) A complete account of obsta- 
cles and resources decreases the 
likelihood .that either pitfalls or . 
potentials will be overlooked in 
the development 6f>a project. • 

(c) Using knowledge of the inf lu- 

, . enoes in the project's environment , 
helps to capitalize on opportuni- 
ties or arrangements that go . 
beyond the resources •mdet a ptoj- 
1 ,ect's direct control, (d) Alter- 
, native strategies or plans, to 
implement any intervention can be 
'created and assessed in the con- 
text of the forcef ield. Careful— 
attention fco the forcef ield sur- 
< rounding a^research project 
. » increases the likelihood that both 

interventions and research designs 
will be implemented as intended. 

Because initial analyses of a 
forcef ield may be -objectively 
incorrect, because perceptions ** 
change over time, and because the 
action of a project may alter its. 
forcef ield, the dynamic, nature of 
the field is to be expected. A 
sensible practice^ therfore, is to 

ERJC 
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renew forcefigld analysis periodi- 
cally, especially when any stra- 
tegy being executed on the basis 
of an initial forcef ield analysis 
is not working well. Breakdowns 
in experimental design or in the 
' irplementation of an intervention 
would both required renewed analy- 
sis. 

* For nore discussion of force- 
field analysis and examples of its 
application see Hersey and Blanch- 
ard (1982, pp. 115-119, 269-272) . 
Practical guidance on working with 
an organization to -analyze its 
,forcefield is provided by Blahton 
& Alley (Note 5., . 103-113) . 

Strategies' — < 

Strategies are plans. „ ^Cord- 
ing to the IDE method, strategies 
are developed from a forcef ield 
analysis, just as objectives and 
interventions derive from a theory 
of action about a problem. Sev- 
eral possible strategies for 
implementing a research project or 
one, of its component interventions 
are likely to exist. The task for 
project implementers and those who 
are attempting to facilitate stra- 
tegy development is to create a' 
plan that is perceived &s feasible 
and attractive. If a critical 
path. in, some plan is blocked and 
no way around th§ obstacle is per- 
ceived, the plan is not a good 
one. Alternative paths that 
objectively exist 'but have not 
been perceived will not be fol- 
lowed. (This point illustrates 
why thorough and Creative force- 
field analysis is helpful.) A 
strategy that appears workable 
will make use of an organizations 
resources to overcome the obsta- 
cles to implementation and 
research. Such a strategy may 
involve (a) moving around, ari < \ 
obstacle, (b) decreasing the 
"strength of. the forces working 
against inplementation, (c) turiv-^ 
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jlng. an obstacle into a resource , 
^or*d) using a strategy in which 
;the dbstacle is irrelevant and 
jieed ,not be overcome. 

w 

• A fully articulated strategy is 
^composed of three kinds of ele- 
ments; critical benchmarks, 

implementation standards , and 

tasks. 

Critical benchmarks . A criti- 
cal benchmark is a key decision f 
agreement , action, or arrangement 
necessary to move forward with a 
plan. A benchmark is much like a 
gate that must be opened to move 
along a path. If the gate does 
not open, progress in executing 
the strategy is blocked. The 
, locations of these benchmarks (or 4 * 
the nature of than) are made clear 
in the process of analyzing the 
forcef ield around an intervention. 
For example , the forcef ield analy- 
sis about a project's efforts tp 
provide in-service graining for 
teachers might imply that* an 
obstacle lies in teacher unwill- 
ingness to participate in training 
outsider of normal/ working hours, 
and that a resource is the author- 
ity of the deputy superintendent 
of schools to grant release time 
and to allocate the funds fo'r sub- 
stitute instructors < The deputy 
superintendent 1 s agreement to 
grant release time athd to author- 
ize the expenditure for substitute 
teachers would then become a- cri- 
tical benchmark. The deputy 
superintendent is a gatekeeper * 
(Lewin,*L947, p. 333) whose psy- , 
chology must be examined to learn 
how to get 1 the gate opened. * 



Specifying when a critical 

* benchmark is to be accomplished 
facilitates management. Any stra- 
tegy will require a tenporal or - 

m logical sequence of milestones 

* that must be^t. In the forego- 
ing example; a failure to accom- - 
plish the critical benchmark would 



signal the neeql to devise a 'new.; 
strategy for getting tfcie training " 
done r °r the need to seek an 
alternative to training. 

Implementation' standards. The 
second part of a strategy for the 
implementation of both interven- 
tion and research are a set of 
implementation standard^. Sound 
research practice has long 
involved attention to survey 
response rates, the measurement of 
independent and dependent varia- 
bles f and the procedures used to 
induce experimental manipula-- 
tions. Researchers in experimen- 
tal social psychology and physiol- 
ogy routinely conduct manipulation 
checks to determine that implemen- 
tation standards have been met. 
In research on the effects of die- 
tary supplementation or deficien- 
cies for example , investigators do 
not Assume that an artificial diet 
has been composed as intended, but 
perform quantitative -analyse^ on 
samples,, of the actual food to det- 
ermine that the diet oonsumea sos 
as ^intended (e.g. f Vohra, Gott- 
frteftsonr^ Kratzer, 19&8)- v And, 
careful researchers on labeling 
effects perform manipulation . 
Checks to see that interpersonal 
expectancies haye been experimen- 
tally modified as intended (e.g. , 
Edterc.fc Shani, 1982) . 

, Inplementation standards in a 
prevention research project might 
incliide the specification of. such 
intervention characteristics as 
(a) the skills, knowledge, and <9 
numbers of staff; (b) the fre^ 
que?!^, 'duration, and content 0 of 
interactions. of workers with 
clients , families, 6r community 
organizations; (c) the specific 
actions to be taken in a range 1 of 
specified situations; an<3 
(d) guidelines for the nature and 
value of reinforcers to be applied 
for specific kinds of performance. 
In general, Inplementation 
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standards should be sufficiently 
concrete and specific that fhey^ 
allow for a coirparison of vhafe is 
being donfe with what is intended. 
• • 
laskS. The third part of a 

' strategy is the set of tasks 
required to execute it. A task 
statement specifies jtfhG will do 

fc Sihat by xb£&. Specifying a person 
to be responsible for executing a 
particular task f even when a group 
will be involved f promotes clar-^ 
ity. And specifying when a task' 
is expected to be completed is an 
additional nanageroent tool. 

Critical benchmarks f iirplemen- 
tation standards f and tasks all 
serve important functions in proj- 
ect management and worker rein- 
forcement: They serve to guide an 
organization 1 s ef forts f and they 
provide one kind of objective 
standard of achievement. A lack 
of such objective standards 
"deprives the workers . ^ . of . 
their legitimate desire for satis- 
faction on a realistic basis. 
JJnder these circumstances, satis~ 
/taction or dissatisfaction with 
V> {one 1 s) ^ own achievement becomes 
mainly a question of temperament" 1 
(Lewin f 1946 f p. 35). 

Development 

1 i^t the very heart of thfe IDE 
1 method 1 is the expectation that 

project "development' and research 
; will be au.pngoing process r _and 
* that the projects environment, is 
dynamic. Only an 'effete organiza- 
tion is immobile f at equilibrium, 
^tension/ reassessment/ .review , 
**j^anfiing f and changes in actions* 
Ttak^h are the hallmarks of vigo- 
. reus projects. Consequently/ IDE 
is a cyclical process of action 
research as progress is made 
towards achieving goals and cbje'o- 
tives (or as goals and objectives 
are redefined) r as new information 
becomes available , as the 
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environment changes , and $s new 
research questions emerge. 

Development occurs largely^ , 
through the use of information. 
Information about the achievement 
Oi. nonachieyement of critical 
benchmarks signals that the force- 
field has been, usefully under- 
stood/ or that developmental 
efforC is required to reassess the 
organization's forcefield. Infor- 
mation that an objective is being 

Achieved signals that an interven- 
tion is effective f and information 
that an objective is not being 

1 achieved signals a reconsideration 
of the appropriateness/ strength/ 
or fidelity .of the intervention^ 
ancL prompts new planning. Infor- 
mation that there is progr<Sss 
towards a goal signals that the 
organization is on the right 
track. Information that there is 
no progress towards the goal may 
signal several things/ depending 
on the. pattern 'Of others feedback. 
If interventions are being imple- 
mented as intended and they are 
achieving their objectives/ the 
theory is called into question. 
If objectives are not being iriet f 
either thq theory or integrity of 

_the intervention/ or both/ should 
be^ scrutinized. Success_in brjLng- 
irig about elusive objectives and 
solving serious problems is not to 

! be expected at once. Bjixt the IDE 
structure is intended to provide 
" interim feedback on, progress to 
" enable a strengthening of the 
project. r Evidence of project 
. effectiveness lends support to the 
theory guiding the project. 

Evaluation 

The IDE structure is intended 
to facilitate several kinds of 
evaluation. The explication of a 
theory of action allows an assess- 
ment of its plausibility/ and an 
assessment of the plausibility or 
strength of the project's planned, 
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interventions in light of the 
theory. m 

By tracking the achievement of 
, critical benchmarks, the structure 
allows assessment of progress 
towards implementing an interven- 
tion. Monitoring implementation 
* standards provides for the assess- 
ment of how yell interventions are 
being executed — it provides mani- 
pulation checks for £oth research- 
ers and project managers. These 
are key elements of formative 
evaluation and the conduct, of 
research. , 

The.FDE methqd is also intended 
to facilitate rigorous summative ' 
evaluation — it promotes experimen- 
tal action research (Chein, Cook, ( 
& Harding, 1948) or at least qua- , 
si-experimental action research. 
It repeatedly /asks the question, 
"How 3o we know the intervention 
made ,the difference?" The imple- 
mentation of an evaluation design' . 
is treated in the same way a£ the 
lirpiementation of any other inter- 
vention. The FDE method assumes 
that evaluation is an essential 

- component of e$£ect$ve project 
development and should receive 
coordinate effort with other- 
aspects of project implementation.' 
Therefore t fospefield analysis is 

- performed for design and daita col- 
lection issues just as it is for 
any other project component. 

. Because project implemented are 
involved in the research ^design 
.and in the specification fi£ the . 
research questions, their commits 
ment to strong evaluation i$ 
expected to* increase. And, 
because the forcefield analysis 
—focuses on the project implement-, 
ers* own, forcefield— their percep- 
tions of" th$ possible — the immedi- 
ate environment of the evaluation 
is tiaken into account when the 
• evaluation , is designed, perhaps 
mitigating, some of the resj.st^nce^ 
to evaluation activities coipmonly 



encountered among implementers. 
Strength and Fideli ty 

Th£ EDE method makes possible 
the assessment of the^'strength and 
ficfelity of planned interventions 
through judgments about theoreti-' 
cal plausibility and benchmark 
monitoring. Hi is assessment can 
occur in two ways. First, m project 
implementers can assess the con- 

' sistency of' their interventions 
and objectives with the theory of 
action underlying their project. 
That is, a project implementer can 
determine whether the .objectives 
sought accord with the theory, and 
whether the interventions planned 
will plausibly achieve the proj- 
ect's objectives. In short, 
theory is a tenplate for making, 
judgments about the appropriate- 
ness o£ interventions and ebjee- , 
tives that project ifitplementers % 
can use. to quality control their, 
own projects. Second, observers 
of a project, including research- 
ers, can assess its a PEiQli 
strength by determining whether 
the planned interventions will 
plausibly lead to the "objectives ^ 

. .or goals' of the project, by ..assess- 
. ing them in comparison to state-* 
of- the art theories in the field 
- and the history of similar proj- 
. ects that have been conducted ,in 

1 the past. 

As a project develops decisions 
to adopt an innovation will have 
been made, md the nature of the 
events important to the project 
- changes. Early in a project's * 
development, major issues. will 
h&ve to do with ideas for inter- 
ventions or With strategies for 
getting an innovation adopted. At 
^ater points, important issues 
will involve the integrity of 'the 
intervention's iitplementation and 
the assessment of effectiveness. 
In the early stages of t npvement 
toward the adoption of an 



innovation, critical benchmarks 
will involve events related to the 
decision to adopt a change. 
Later, 'implementation standards or * 
quality control checks on the 
faithfulness of the intplementatidn 
will increase in importance. 

The documentation and assess- 
ment of interventions as imple- 
mented may involve detailed manu- 
als for the administration of 
. treatments or programs; descrip- 
tions of the characteristics of 
staff and target groups; and 
accounts of the duration and sche- 
duling of ..treatments or events, 
treatment protocols, or proportion 
of the population served. .Thfere- 
fore, as* projects deyelop, EDE 
increasingly focuses on the devel- ^ 
opment of manuals to guide service 
delivery, make diagnoses, and 
train the staff. Action research 
< projects using this nrx3el will 
develop strategies to monitor 
staff performance, provide incen- 
, tive structures to keep perf or- 
. mance according to specifications, 
and the like. 'The implementation 
of those strategies is jexpected to 
have two consequences:*' (a) the 
plans and their .execution- would 
increase the integrity ,o£ the . 
, ...'Intervention, and (b) th§ infbrma- 
by* the implementa- 
tion of these plans would describe 
the integrity of .interventions. 
\ Manuals developed in the course of 
carefully ^implementing a project 
. will -allow for subsequent close 
replications in future research or 
application. 

SQSSi Examples 

Two actual exanples of projects 
now in$ej3way provide insight into, 
the Way tjie HJE method enables- the 
• translation of practitioner ideas 
* into theoretical terms and pro- 
' notes project development., First, 
consider an example of a delin- 
tj^uncy prevention project 
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• * 
involving a, peer group interven- * 
tion. Basically, the project 
iitplementers assume that delin- 
quent behavior is supported by a, 
normative subcaDbure. To prevent 
delinquency the project intended 
to compose groups of delinquent 
and non-delinquent youths and to 
structure group interaction such 
that delinquent behavior and 
expressions of beliefs counter to 
traditional moral beliefs are con- 
fronted in the group. Altering 
beliefs is an impor^tpnt objective. 
This project's theoretical 
rational and statement of inter- 
vention standards evolved over a 
period of over two years^in which 
a field worker for the evaluation 
(Jjane St. John) and project imple- 
mented (Sonny Luster and William 
Kottman) used the EDE method. 
Early research results' for this** % v 
project implied that theoretical 
intermediary objectives #ere not 
always being. met. In early , 
results 3 measure of belief in 
conventional social rules was ■ t 
negatively effected by the treat- 
ment for non-delinquent group par- 
ticipants. This research outcome 
wafe used by the project implement- 
ed to revise their treatment spe- 
cifications and to devise? ways to„ 
monitor group process ino re 
closely. Specifically, group 
leaders were provided additional 
training, more attention was given 
to the composition of the groups, 
and monitoring of grdup interac- 
tion using the Hill X1977; Note 6) 
" interaction matrix was initiated. 
Although the development and eval- 
uation of this project is still 
underway, the early negative 
result has disappeared in subse- 
quent evaluations. 

A second actual example is a^ 
project currently being conducted 
by a Southern urban school system. 
Essentially/ this project's theory 
assumes that assistance and pres- 
sure to inprove academic 
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performance, 'contained with the r l 
opportunity 60 develop attachment^ 
to prosoeial' others, will result 7 
in more rewarding experiences in 
school , greater attachment, and 
less truancy and delinquent behav- 
ior. This theory suggests that 
students experiencing academic 
difficulties and who are disrup- 
tive in school ^iould receive spe- 
cial attention from project per- 
sonnel in the form of tutoring and 
advocacy on behalf of the student 
with teachers, parents, and oth- 
ers. The evaluation, field worker 
(Denise Gottf redson) and project 
staff (Doris Cpaxum, Barbara Dil- 
ligard, Ann Birdseye, and Martha 
Stuart) used the IDE'nethod. * . 
Early implementation nonitoring 
showed that the intervention was 
not beirig implemented as intended, 
with project personnel interacting 
mostly with' students who did not 
fall .within the. target group, and 
interacting only, to a .limited 
extent with target* students. 
Clearer implementatipn standards 
were specified for the workers to 
follow,' monitoring of these**fetan- 
dards increased, and subsequeht 
^research results .^Wrintproved 
inpl orientation and have begun to 
show increased' academic perfor- 
mance as measured both by stan- 
- dardized test scores and school 
grades, and some evidence of" ' 
decreased school .disruptiveness 
♦and misconduct when randomly equi- 
valent treatment and control stu- 
dents are compared. Some 
intermediary outcomes, notably 
attachment to school, yere not 
affected as expected, suggesting 
that the program can be strength- 
: ened further. 



Limitations and Potential ' & 

Criticisms 



The EDE . method has some limita- 
tion, creates some tensions, and 
is open to. criticism. The most ' 



inpo rtant^ limitations ap^ar to be 
that it* is complex, it is time and 
expertise intensive, it fails to 
completely resolye the tensions 
sumraative ^evaluation causes for 
project iinplementers, it is an 
iniperfect mechanism for coping 1 
with the separate goals of project 
sponsors and implementing organi- 
zations when these are not com- 
pletely in -accord, and it con- 
fronts researchers and * ■ 
impiementers with tough decisions 
involving the sacrifice of rigor- 
ous research designs in order to 
achieve some aspe<ie of project 
implementation . 

The human behavior required to 
successfully implement the IDE 
method is complex, and the meth- 
od 1 s implementation calls for an 
investment of human resources. 
Use of the IDE structure calls for* 
high levels of interpersonal com- 
petency f tact, patience, cx)mrruni- 
cation skill, and tnderstanding of ' 
group relations in organizations.* 
In addition, 'it calls for* a tho--^. 
rough . mderStanding of evaluation 5 
methods--measurement, social sci- 
ence theory, experimental and qua- 
si-experimental methods, and 

" statistics. Ironically, this com- 
bination of competencies are 

. rarely found in one and the same 
person, suggesting that a team of 
workers may be required to imple- 
ment this approach to organiza- 
tional change. Furthermore, the . 
cyclical or developmental nature" 
•of PDE requires constant (or at 

1 least frequent periodic) atten- - 
tion, monitoring, updating, and 
information (^mmunifcation. This 
taxing, process' goes beypnd the 
effort typically expended in 
research. 

* ' Seme trade-offs are likely to 
be involvefflj^^sEaf fing an action 
Research effort using the HIE 



nte'thod. Because staff with 'the 
requisite skills are hard to find 1 
in a typical organization train- 
ing wiiitbe required, ]&cperience 
implies that social sgientists 
trained primarily in research 
irethodology, statistics, andy 
theory can successfully implement, 
the method, but that thetf reqhise 
additional training in organiza- 
tion development to do so.' 

Tensions 

Tension appears, endemic in ; 
evaluation research. Tbo often in 
the pastj evaluation has been used 
as a tool for canceling a proj- , 
ect — even when positive evalua- 
tions could not reasonably be 1 
expected at an early stage of 
project development. .Tension is 
also created by the inherently 
political environment of action 
projects, and by environments 
where the successful project does 
not rock the boat. The develop- 
ment of sound programs usually 
requires the expenditure of time 
arid money ,\and often implies the 
necessity of arrangements that are 
disruptive. 

Tradeoffs and Research Rigor 

Program Development Evaluation 
is value laden. Ifcrticipation of 
project implemented is a funda- 
mental principle in the IDE pro- 
cess; pursuit. of the goals and . 
Objectives of the implementing 
organization are generally assumed 
to be desirable (although open to 
question) . Furthermore , an. aim of 
PDE is to develop the implementing 
organization's capacity to accom- 
plish its:goals. ^Therefore, 
researchers and irrplementers col- 
laborate in evaluation design, • 
question formulation , and pan- 
ning. - As a result, researchers 
extensively intervene in project 
development— indeed they become a 
part of the proj ( ect. 
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Sane evali^tors (Perloff, 1979)' 
see this as undesirable in a sum- 
ma tive evaluation because it ' 
raises' questions about the gener- 
alizability^of the results to 
situations where researchers are 
absent. . In addition, just as" 
research heeds sometimes intrude 
in 'project operations byvcreating 
new tasks or structural arrange- ' 
merits, the pursuit of a project's 

. programmatic activities can result 
in compromises in research design. 
As Deutsch (1968, p. 466) says, 
"Ihe, danger that confronts the 
research worker in such situations 
is the possibility that his 

. research design or methodology . 
wiliLbe sacrificed to the achieve- 
ment of the social-action objec- 
tive." 

^ This "danger" nay account in 
part for the relucjtance academic 
social scientists have shewn to 
participate in action research. * 
This danger seeps a sn&Ll price to 
pay in exchange - for the opportu- 
nity action research .creates to 
contribute to -the solution* of 
social 'problems, although the 
tradeoffs involving evaluation 
rigor ate painful.. In short,' the 
PEE. method is no panacea for this 
tough prbblem.' << - 

CQnpiexitv 

> The EDE method is complex. 
Unfortunately, each component of 
the method seems at present to be 
useful and' desirable in action 
reserch efforts. Nevertheless, 
this complexity suggests that a 
more streamlined jnethod is appro- 
priate when doi^g, short-term 
organizatipnal cfevelopment inter- 
ventions that do not have a < 
research, purpose. Consequently, 
for many btief organization devel- 
opment interventions, the selec- 
tive use of those portions of the' 
PDE structure that seem to be the 
most relevant for the problem at 
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hand is more appropriate than 
attempting to use all parts of the 
method. Researchers should not 
abandon any part of tl)e methpd. , 

Practical Application, 

Experience implies, that one or 
another part of this process is 
useful to project implemehters;in 
defining their cwn jobs, in formu- 
lating plans, and in clarifying 
their intentions. Experience also 
implies that the entire process is > 
sometimes viewed as burdensome. 
Omtfte whole > this structure seems 
a «clear improvement over some more 
traditional research methods 
because it involves implementers 
in research planning, because it 
explicitly attempts, to build sum- 
mative evaluation structures based 
on an organization's forcefield, 
and because it focuses on goals 
and objectives of cCncern to 
implertenters. 

My colleagues and I are using 
the IDE method in the 'theory-rid- 
den evaluation of delinquency 
projects conducted by 17 different 
organizations. In an area where, 
true experiments are almost unk- 
nown (Dixon & Wright, l975;,Gott- 
fredson,;i981; Ogawa, Note .70, six 
of these projects are iruplementing 
true experiments. CXrr opinion is 

• that the quasi-experimental 
designs of most of' the remaining 
projects ane substantially better 

■ than would have been the case had 
project implementers not been 
involved in, an evaluation using 
the IDE method. Even where the * 
original research design has bro- 
ken down, project implementers 
have usually sh^re&'with the 
rese&nchers the task of creating 
alternative designs and in devel- 
oping plans to prevent subsequent 
breakdowns. 1 

The greatest virtues of the IDE 
. method appear to be (a) its 



ability to make possible theoreti- 
cal tests through research on 
action projects, (bNjlts ability 
to elaborate clear measurable 
intermediary outcomes, or <£)jec- . 

♦ tives, useful in assessing the^.^ 
effectiveness of interventions nT 
theoretical terms, ic} its ability 
to provide project implemepters - 
with the "tools to assess their cwn 
efforts by measuring interventions 

f against theory and objectives, 
(d) ' its ability jto generate crea- 
tive strategies pefcceived as feas- 
ible to implementers based on the , 
divergent and then corfrergent 
thinking that .takes* £Lace 'ir> • 
forcefield analysis, '(e) its abil- 
ity to .involve project implement- 
ers in the research enterprise by 
engaging them in the specification 

^^J3£ measurable goals and objec- 
tives, and in the Creation of 
evaluation designs, (f) its abil- 
ity to provide short-term assess- , 
ments.of progress through the 

• monitoring , of critical benchmarks, 
implementation standards, and 
tasks, (g) its ability to enable 

- researchers to understand the 
nature o£ a project by translating 
in^lementerVs. ideas into a struc- 
ture language .of theoretical 
-research, and (h) its ability to 
serve as a structure for communi- 
cation between researchers and 
practitioners. 

PEE makes serious' organiza-^ 
t^ional change $nd rigorous 
research more attractive to organ- 
izations despite dts inability tcr 
make it' truly* palatable to all of 
them. Ideally, practice and * 
research would be merged into a 
single enterprise in which rigor- 
ous theoretical "research becomes 
art integral component of program 
operation. It 1 is unrealistic to 
think ihat most practitioners will 
> eve'r acquire all 'bf the technical 
skills required to' systematically 
conduct rigorous research on their 
activities (just as in is 
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unrealistic to expect most 1 
researchers to become adroit prac- 
titioners) . In addition, truly 
rigorous research is not-always 
called for in the; deveiopnent of a 
projects and not all action proj- 
ects test theoretically interest- 
ing ideas. Wh^n rigorous evalua- 



fcion is called for, however /.the 
PDE structure is helpful. Program 
' Development Evaluation does not 
successfully resolve many of the 
sources- bf tension in merging 
action with research , but it is 
^progress. 

* -# 
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